Versatile Machine Learning Evaluation Metrics Toolbox
Oct 2021 ~ FedEx Express
Length: 1,5mo (at 0.5 FTE)
Programming language: Python (Pandas, NumPy, Matplotlib, seaborn,
scikit-learn, os, time)
Data: sklearn datasets generated for testing purposes
Problem description:
Program a class in OOP fashion that contains several methods for evaluating the
performance of Machine Learning models
Approach & Results:
As there are multiple Machine Learning models used at FedEx, the developers could
benefit from a class consisting of functions for ML model performance evaluation. With such
a module available, the evaluation procedure could be simplified and standardized, while the
Data Science team could save time and make well-informed decisions about model training.
Accordingly, a class was built to include several
evaluation functions ranging from standard metrics (e.g. Accuracy, Recall, F1, FNR), to
curve plotting (e.g. Learning curve, Validation curve, Precision-Recall curve, ROC) and
feature importance (e.g. tree models, permutation importance), by leveraging the scikit-learn library.
Also, an extensive documentation
was written to ease the functions utilization, followed by error handling to make the module
forestall anticipated errors if possible or provide the user insights into what went wrong.